Skip to content

feat(dune_lang): format multi-line strings - #13758

Draft
Alizter wants to merge 1 commit into
ocaml:mainfrom
Alizter:push-kxrkwxtlkorp
Draft

feat(dune_lang): format multi-line strings#13758
Alizter wants to merge 1 commit into
ocaml:mainfrom
Alizter:push-kxrkwxtlkorp

Conversation

@Alizter

@Alizter Alizter commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Preserve multiline block-string syntax when formatting Dune files that use language version 3.25 or later. Both escaped (\|) and raw (\>) block lines are retained whenever block syntax can represent the original string or template exactly.

The formatter preserves:

  • literal contents and newline semantics, including CRLF and EOF-terminated strings;
  • the distinction between raw and escaped continuation lines;
  • pform expansion and escaping; and
  • formatting idempotence for nested and adjacent block strings.

For language versions before 3.25, or when block syntax cannot preserve the original semantics, the formatter emits an ordinary quoted string or template instead.

Implementation

AST and CST parsing

Block strings are concrete syntax, so the AST continues to represent only their meaning as an ordinary quoted string or quoted template. Previously, AST and CST parsing shared one AST-shaped intermediate tree, with comments represented by synthetic AST nodes. That representation cannot retain per-line block syntax without exposing it in the AST or adding another synthetic encoding. The parser therefore now has direct AST and CST loops: this intentional structural duplication keeps the AST API and hot path unchanged while allowing the CST to retain block metadata without an intermediate tree.

Lexer

The lexer retains block metadata only in CST mode. AST mode continues to emit the existing quoted-string or template tokens, while CST mode emits a block-string token containing each logical line's raw/escaped kind and template parts. Its block state distinguishes semantic newlines from physical continuations and handles escapes, pforms, and CRLF. Separate quoted-string and block-string token/CST constructors make invalid mixed states unrepresentable.

Testing

Expect tests cover parsing, CST-to-AST conversion, semantic round trips, and formatting idempotence. Blackbox tests cover formatter output and existing pform/newline behavior. The baseline behavior tests landed separately in #16201 and #16202.

Related work

Supersedes #10780 and #11581.

@Alizter
Alizter marked this pull request as draft March 9, 2026 22:19
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 71caf09 to 7c16a47 Compare March 9, 2026 22:22
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 7c16a47 to 736212e Compare March 18, 2026 00:02
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 736212e to 241c9af Compare March 18, 2026 00:03
@Alizter
Alizter requested review from nojb and rgrinberg March 18, 2026 00:04
@Alizter
Alizter marked this pull request as ready for review March 18, 2026 00:05
@Alizter
Alizter removed request for nojb and rgrinberg March 18, 2026 00:06
@Alizter
Alizter marked this pull request as draft March 18, 2026 00:06
@Alizter

Alizter commented Mar 18, 2026

Copy link
Copy Markdown
Collaborator Author

I think I will iterate on this a bit further, I'm still not happy with it myself. But others are welcome to take a look.

@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 241c9af to 9f52c4e Compare March 18, 2026 00:59
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch 2 times, most recently from 3e75415 to 2c37395 Compare April 19, 2026 19:56
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch 3 times, most recently from c93244f to 22b6e2a Compare April 30, 2026 21:32
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 22b6e2a to 726f5dc Compare August 25, 2026 13:30
Alizter added a commit that referenced this pull request Aug 25, 2026
## Description

Add regression coverage for pforms in block strings. Pform-like text in
raw block strings remains literal, while escaped block strings expand
pforms normally.

## Motivation

This locks down existing parsing semantics ahead of the formatter work
in #13758. Split from #13758 for independent review.
Alizter added a commit that referenced this pull request Aug 25, 2026
## Description

Add regression coverage showing that an escaped `\n` and a literal
newline between block-string lines produce the same string value.
Distinct action markers ensure both forms execute during the test.

## Motivation

This locks down existing parsing semantics ahead of the formatter work
in #13758. Split from #13758 for independent review.
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch 3 times, most recently from 8b0b91f to 03ff593 Compare August 25, 2026 17:31
Preserve escaped ("\|") and raw ("\>") block-string syntax when
formatting Dune files with language version 3.25 or later, without
changing parsed string values or pform expansion semantics.

For older language versions, format block strings as regular quoted
strings or templates.

Supersedes ocaml#10780 and ocaml#11581.

Signed-off-by: Ali Caglayan <alizter@gmail.com>
@Alizter
Alizter force-pushed the push-kxrkwxtlkorp branch from 03ff593 to fab90e2 Compare August 25, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant